Papers with FEVER dataset
Unsupervised Question Answering for Fact-Checking (D19-66)
Copied to clipboard
| Challenge: | Recent Deep Learning (DL) models have achieved human-level accuracy on natural language tasks such as question-answering, natural language inference, and textual entailment. |
| Approach: | They propose an unsupervised question-answering based approach for a similar task, fact-checking. |
| Outcome: | The proposed approach achieves label accuracy of 80.2% on the development set and 80.25% on the test set. |
Zero-shot Fact Verification by Claim Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification require large datasets, which can be expensive. |
| Approach: | They propose a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. |
| Outcome: | The proposed framework reduces the demand for human-annotated training data and improves a model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples. |
GLAF: Global-to-Local Aggregation and Fission Network for Semantic Level Fact Verification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing fact verification models lack fine-grained reasoning over key entities . GLAF uses local fission reasoning to capture latent logical relations between clues . |
| Approach: | They propose a global-to-local fission and fissional network to capture latent logical relations hidden in multiple evidence clues. |
| Outcome: | The proposed network achieves state-of-the-art on a FEVER dataset with a 77.62% FEVER score. |
Connecting the Dots Between Fact Verification and Fake News Detection (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for detecting fake news rely heavily on supervised learning on a large scale dataset with news articles labeled as fake or real by human experts. |
| Approach: | They propose a simple yet effective approach to connect the dots between fact verification and fake news detection by using a text summarization model pre-trained on news corpora to summarize the long news article into a short claim. |
| Outcome: | The proposed approach enables zero-shot fake news detection, alleviating the need for large scale training data to train fake news detector models. |
AttentiveChecker: A Bi-Directional Attention Flow Mechanism for Fact Verification (N19-1)
Copied to clipboard
| Challenge: | Existing fact-checking datasets do not provide manual annotations for sentence-level evidence. |
| Approach: | They propose a task-agnostic pipelined system that extracts textual evidence that supports or refutes a factual claim from Wikipedia pages. |
| Outcome: | The proposed system achieves state-of-the-art results on the FEVER dataset. |
Towards Debiasing Fact Verification Models (D19-1)
Copied to clipboard
Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, Regina Barzilay
| Challenge: | Prior research has shown that data collection methods that use crowdsourcing introduce idiosyncratic biases that impact performance in unexpected ways. |
| Approach: | They propose a method to regularize the training data to avoid idiosyncrasies in the datasets that are used for fact verification. |
| Outcome: | The proposed model outperforms the existing model on the FEVER dataset, achieving 61.7% of the baseline. |
Evidence Retrieval is almost All You Need for Fact Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evidence retrieval methods adopt a trivial retrieval strategy, resulting in task-irrelevant evidence and undesirable performance. |
| Approach: | They propose a framework for evidence retrieval and joint fact verification that integrates two modules. |
| Outcome: | The proposed framework improves evidence retrieval and claims verification on a FEVER dataset. |
Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction (2023.findings-acl)
Copied to clipboard
| Challenge: | a novel latent variable model for fact-checking and analysis learns to identify the veracity of a claim and its relevant evidences. |
| Approach: | They propose to disentangle the per-evidence relevance probability and its contribution to the final veracity probability in an interpretable way. |
| Outcome: | The proposed model can achieve competitive results on the FEVER dataset while using significantly fewer parameters. |
Towards the Necessity for Debiasing Natural Language Inference Datasets (2020.lrec-1)
Copied to clipboard
| Challenge: | Delexicalization of datasets can improve natural language inference performance . a dataset with a delexicalized version of the FEVER dataset is used for natural language learning . |
| Approach: | They propose two techniques for delexicalization that modify annotated datasets to control the importance of lexical entities. |
| Outcome: | The proposed methods maintain performance in-domain and improve performance in some out-of-domain settings. |